Medical imaging techniques are used in the detection of cancer or precancerous conditions in a patient. An important application is in the detection of tumors or potential tumors in breast cancer. Potential tumors are difficult to detect but it is known that such tissue typically exhibits a more rapid intake (wash-in) of contrast agent, as well as a more rapid washout than adjacent, non-tumor tissue. This difference in behavior allows the detection of suspect tissue through comparison of images of a patient made before and after such wash-in and/or washout. Using such time sequential images made by an imaging technique such as magnetic resonance imaging (MRI), a comparison may be made between images to detect differences due to the contrast wash-in and washout behavior exhibit by different regions of the acquired MR volume so as to detect such suspect tissue.
In order to perform this detection advantageously, one needs to track the intensity of a single voxel in a temporal sequence of such volumes. However, a difficulty arises in that the patient typically moves between consecutive acquisitions and thereby introduces motion-related differences between the acquired images whereby a single point in space can no longer be tracked, unless motion correction is performed.
It is an object of the present invention to solve the motion correction problem in an advantageous manner in, for example, breast MR detection of potential tumors which are detected as tissue that has a rapid intake (wash-in) of contrast agent, as well as a rapid washout.
Prior art approaches to solving this problem in the past have computed the optic-flow between two images, of which an arbitrary one is selected as the reference among the images of the sequence. The two images are obtained from the acquired images by computing a Laplacian pyramid. The optic flow is calculated by solving a minimization problem of the point-to-point difference between the two Laplacian images.
The problem of estimating the geometric deformation between two images has a long history in the scientific literature. Techniques for computing the optic flow can be traced back to papers like B. K. Horn and B. G. Schunk: Determining optical flow, Artificial Intelligence, 17:185-203, 1981, and references cited therein. The use of the cross-correlation as similarity measure can be found in Olivier Faugeras, Bernard Hotz, Herv Mathieu, Thierry Viville, Zhengyou Zhang, Pascal Fua, Eric Thron, Laurent Moll, Grard Berry, Jean Vuillemin, Patrice Bertin, and Catherine Proy: Real time correlation based stereo: algorithm implementations and applications, Technical Report 2013, INRIA Sophia-Antipolis, France, 1993; Olivier Faugeras and Renaud Keriven: Variational principles, surface evolution, PDE's, level set methods and the stereo problem, IEEE Transactions on Image Processing, 7(3):336-344, March 1998; Jacques Bride and Gerardo Hermosillo: Recalage rigide sans contrainte de preservation d'intensite par regression heteroscdastique. In TAIMA, Hammamet, Tunisie, October 2001; P Cachier and X. Pennec: 3d non-rigid registration by gradient descent on a gaussian weighted similarity measure using convolutions, in Proceedings of MMBIA, pages 182-189, June 2000; and T. Netsch, P. Rosch, A. van Muiswinkel, and J. Weese: Towards real-time multi-modality 3d medical image registration, in Proceedings of the 8th International Conference on ComputerVision, Vancouver, Canada, 2001. IEEE Computer Society, IEEE Computer Society Press.
Other related similarity measures have been proposed, like the correlation ratio, A. Roche, G. Malandain, X. Pennec, and N. Ayache: The correlation ratio as new similarity metric for multimodal image registration, in W. M. Wells III, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis: Multi-modal volume registration by maximization of mutual information. Medical Image Analysis, 1(1):35-51, 1996, pages 1115-1124, and the mutual information, Paul Viola: Alignment by Maximisation of Mutual Information. PhD thesis, MIT, 1995; Paul Viola and William M. Wells III: Alignment by maximization of mutual information, The International Journal of Computer Vision, 24(2): 137-154, 1997; F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens: Multimodality image registration by maximization of mutual information, IEEE transactions on Medical Imaging, 16(2):187-198, April 1997; W. M. Wells III et al., op. cit., among others, R. P. Woods, J. C. Maziotta, and S. R. Cherry: MRI-pet registration with automated algorithm, Journal of computer assisted tomography, 17(4):536-546, 1993; D. Hill: Combination of 3D medical images from multiple modalities. PhD thesis, University of London, December 1993: G. Penney, J. Weese, J. A. Little, P. Desmedt, D. LG. Hill, and D. J. Hawkes: A comparison of similarity measures for use in 2d-3d medical image registration, In J.van Leeuwen G. Goos, J. Hartmanis, editor, First International Conference on Medical Image Computing and Computer-Assisted Intervention, volume 1496 of Lecture Notes in Computer Science. Springer, 1998; and M. E. Leventon and W. E. L. Grimson: Multi-Modal Volume Registration Using Joint Intensity Distributions: in W. M. Wells, A. Colchester, and S. Delp, editors. Number 1496 in Lecture Notes in Computer Science, Cambridge, Mass., USA, October 1998. Springer.
Conjugate Gradient minimization is described in William H. Press, Brian P. Flannery, Saul A. Teukolsky, and William T. Vetterling: Numerical Recipes in C. Cambridge University Press, 1988. The type (or family) of deformation which is assumed is the second key component of any motion correction algorithm. Parametric transformations are the most commonly used. See Chuck Meyer, Jennifer Boes, Boklye Kim, and Peyton Bland: Evaluation of control point selection in automatic, mutual information driven, 3d warping, in J.van Leeuwen G. Goos, J. Hartmanis, editor, First International Conference on Medical Image Computing and Computer-Assisted Intervention, Proceedings, volume 1496 of Lecture Notes in Computer Science, October 1998; D. R{umlaut over ( )}uckert, C. Hayes, C. Studholme, P. Summers, M. Leach, and D. J. Hawkes: Non-rigid registration of breast MR images using mutual information, in W. M. Wells, A. Colchester, and S. Delp, editors, Number 1496 in Lecture Notes in Computer Science, Cambridge, Mass., USA, October 1998, Springer; Paul Viola. Alignment by Maximisation of Mutual Information, PhD thesis, MIT, 1995; W. M. Wells III, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis. Multi-modal volume registration by maximization of mutual information. Medical Image Analysis, 1(1):35-51, 1996; and Paul Viola and William M. Wells III: Alignement by maximization of mutual information, The International Journal of Computer Vision, 24(2): 137-154, 1997.
When the deformation is not defined parametrically, the family is often constrained by requiring some smoothness of the displacement field, possibly preserving discontinuities. See J. P. Thirion. Image matching as a diffusion process: An analogy with Maxwell's demons, Medical Image Analysis, 2(3):243-260, 1998; L. Alvarez, R. Deriche, J. Weickert, and J. Sánchez: Dense disparity map estimation respecting image discontinuities: A PDE and scale-space based approach, International Journal of Visual Communication and Image Representation, Special Issue on Partial Differential Equations in Image Processing, Computer Vision and Computer Graphics, 2000; M. Proesmans, L. Van Gool, E. Pauwels, and A. Oosterlinck: Determination of Optical Flow and its Discontinuities using Non-Linear Diffusion, in Proceedings of the 3rd ECCV, II, number 801 in Lecture Notes in Computer Science, pages 295-304, Springer-Verlag, 1994; and L. Alvarez, J. Weickert, and J. Sánchez: Reliable Estimation of Dense Optical Flow Fields with Large Displacements, Technical report, Cuadernos del Instituto Universitario de Ciencias y Tecnologias Ciberneticas, 2000: a revised version has appeared at IJCV 39(1):41-56,2000; E. Mmin and P. Prez: A multigrid approach for hierarchical motion estimation, in Proceedings of the 6th International Conference on Computer Vision, pages 933-938, IEEE Computer Society Press, Bombay, India, January 1998; E. Mmin and P. Prez: Dense/parametric estimation of fluid flows, in IEEE Int. Conf. on Image Processing, ICIP'99, Kobe, Japan, October 1999: G. Aubert, R. Deriche, and P. Kornprobst: Computing optical flow via variational techniques, SIAM Journal of Applied Mathematics, 60(1): 156-182, 1999; G. Aubert and P. Kornprobst: A mathematical study of the relaxed optical flow problem in the space BV, SIAM Journal on Mathematical Analysis, 30(6): 1282-1308, 1999; and R. Deriche, P. Kornprobst, and G. Aubert: Optical flow estimation while preserving its discontinuities: A variational approach, in Proceedings of the 2nd Asian Conference on Computer Vision, volume 2, pages 71-80, Singapore, December 1995.
Some regularizing approaches are based on explicit smoothing of the field, as in J. P. Thirion: Image matching as a diffusion process: An analogy with Maxwell's demons, Medical Image Analysis, 2(3):243-260, 1998; and Christophe Chefd'hotel, Gerardo Hermosillo, and Olivier Faugeras. Flows of Diffeomorphisms for Multimodal Image Registration, in International Symposium on Biomedical Imaging. IEEE, 2002, while others consider an additive term in the error criterion, yielding (possibly anisotropic) diffusion terms, see G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, volume 147 of Applied Mathematical Sciences, Springer-Verlag, January 2002; J. Weickert and C. Schnörr: A theoretical framework for convex regularizers in pde-based computation of image motion, The International Journal of Computer Vision, 45(3):245-264, December 2001; Gerardo Hermosillo, Christophe Chefd'hotel, and Olivier Faugeras: Variational methods for multimodal image matching, The International Journal of Computer Vision, 50(3):329-343, November 2002; G. Hermosillo and O. Faugeras: Dense image matching with global and local statistical criteria: a variational approach, in Proceedings of CVPR'01, 2001; and Gerardo Hermosillo: Variational Methods for Multimodal Image Matching, PhD thesis, INRIA: the document is accessible at ftp://ftp-sop.inria.fr/robotvis/html/Papers/hermosillo:02.ps.gz, 2002.
Fluid methods fix the amount of desired smoothness or fluidness of the deformation using a single parameter. See Christophe Chefd'hotel, Gerardo Hermosillo, and Olivier Faugeras: Flows of Diffeomorphisms for Multimodal Image Registration, in International Symposium on Biomedical Imaging, IEEE, 2002; Gary Christensen, MI Miller, and MW Vannier: A 3D deformable magnetic resonance textbook based on elasticity, in Proceedings of the American Association for Artificial Intelligence, Symposium: Applications of Computer Vision in Medical Image Processing, 1994; and Alain Trouv: Diffeomorphisms groups and pattern matching in image analysis, International Journal of Computer Vision, 28(3):213-21, 1998.
Multi-resolution approaches have also been previously investigated. See L. Alvarez, J. Weickert, and J. Sánchez: Reliable Estimation of Dense Optical Flow Fields with Large Displacements, Technical report, Cuadernos del Instituto Universitario de Ciencias y Tecnologias Ciberneticas, 2000. A revised version has appeared at IJCV 39(1):41-56,2000. In L. Alvarez et al., op. cit., a scale-space focusing strategy is used. Most of the existing methods either do not account for intensity variations or are limited to parametric transformations.
Extensions to more complex transformations which account for intensity variations include approaches relying on block-matching strategies. See J. B. A. Maintz, H. W. Meijering, and M. A. Viergever: General multimodal elastic registration based on mutual information, in Medical Imaging 1998—Image Processing, volume 3338, pages 144-154. SPIE, 1998; T. Gaens, F. Maes D. Vandermeulen, and P. Suetens: Non-rigid multimodal image registration using mutual information, in J.van Leeuwen G. Goos, J. Hartmanis, editor, First International Conference on Medical Image Computing and Computer-Assisted Intervention, volume 1496 of Lecture Notes in Computer Science Springer, 1998; and N. Hata, T. Dohi, S. Warfield, W. Wells III, R. Kikinis, and F. A. Jolesz: Multi-modality deformable registration of pre-and intra-operative images for MRI-guided brain surgery, in J.van Leeuwen G. Goos, J. Hartmanis, editor, First International Conference on Medical Image Computing and Computer-Assisted Intervention, volume 1496 of Lecture Notes in Computer Science. Springer, 1998; or parametric intensity corrections, see A. Roche, A. Guimond, J. Meunier, and N. Ayache: Multimodal Elastic Matching of Brain Images, in Proceedings of the 6th European Conference on Computer Vision, Dublin, Ireland, June 2000.
Some recent approaches rely on the computation of the gradient of the local cross-correlation. See P. Cachier and X. Pennec: 3d non-rigid registration by gradient descent on a gaussian weighted similarity measure using convolutions. In Proceedings of MMBIA, pages 182-189, June 2000; T. Netsch, P. Rosch, A. van Muiswinkel, and J. Weese: Towards real-time multi-modality 3D medical image registration, in Proceedings of the 8th International Conference on Computer Vision, Vancouver, Canada, 2001, IEEE Computer Society, IEEE Computer Society Press; Gerardo Hermosillo, Christophe Chefd'hotel, and Olivier Faugeras. Variational methods for multimodal image matching. The International Journal of Computer Vision, 50(3):329-343, November 2002; G. Hermosillo and O. Faugeras: Dense image matching with global and local statistical criteria: a variational approach, in Proceedings of CVPR'01, 2001; Gerardo Hermosillo: Variational Methods for Multimodal Image Matching. Phd thesis, INRIA: the document is accessible at ftp://ftp-sop.inria.fr/robotvis/html/Papers/hermosillo:02.ps.gz, 2002; and Christophe Chefd'hotel, Gerardo Hermosillo, and Olivier Faugeras: Flows of Diffeomorphisms for Multimodal Image Registration, in International Symposium on Biomedical Imaging. IEEE, 2002.
General background material on optic flow and on image pyramids may be found in textbooks and publications relating to image processing. Textbooks useful in providing background material helpful to gaining a better understanding of the present invention include, for example, FUNDAMENTALS OF IMAGE PROCESSING by Arthur R. Weeks, SPIE Optical Engineering Press & IEEE Press; 1996; IMAGE PROCESSING, ANALYSIS, AND MACHINE VISION, Second Edition, by Milan Sonka et al., PWS Publishing; 1999; and DIGITAL IMAGE PROCESSING, Second Edition, by Rafael C. Gonzalez et al., Prentice Hall; 2002.